Modified backward feature selection by cross validation

Since training of a classifier takes time, usually some criterion other than the recognition rate is used for feature selection. This may, however, leads to deteriorating the generalization ability by feature selection. To overcome this problem, in this paper, we propose modified backward feature selection by cross validation. Initially, we determine the candidate set which consists of the features that do not deteriorate the generalization ability, if each is deleted from the initial set of features. If the generalization ability is not deteriorated even if all the candidate features are deleted, we terminate the algorithm. Otherwise, we delete by backward deletion the candidate feature that improves the generalization ability the most, and determine the candidate set that is a subset of the current candidate set. We iterate the above procedure until the candidate set is empty. We evaluate our method using support vector machines for some benchmark data sets and show that many features are deleted without deteriorating the generalization ability.